Field of the Invention
The present invention generally relates to a neural network, and more particularly, to a control device for controlling the learning in the neural network, which is capable of forcedly updating values of weights of synapse connections of the neural network from outside of the neural network during the learning of the neural network.
Recently, multilayered neural networks which learn by a learning method based on error back-propagation have been utilized in the fields of speech recognition and character recognition. Conventionally, not less than three-layer perceptron type neural networks have been generally used for such fields. As shown in FIG. 2, a three-layer perceptron type neural network is constituted by an input layer 1, an intermediate layer 2 and an output layer 3. The input layer 1 has a plurality of units 5. The intermediate layer 2 has a plurality of units 6, and the output layer 3 has a plurality of units 7. The network is formed by connecting each of the units 5 of the input layer 1 with all the units 6 of the intermediate layer 2 through synapse connections and connecting each of the units 6 of the intermediate layer with all the units 7 of the output layer 3 through synapse connections. When input data are inputted to the units 5 of the input layer 1, output data corresponding to configuration of this network are outputted from the units 7 of the output layer 3.
Each group of units (5 to 7) includes a receiver portion for receiving inputs from a preceding group of units, a conversion portion for converting the inputs into data based upon a predetermined rule and an output portion for outputting the converted data. Weights indicative of the strength of the synapse connections among the units (5 to 7) are imparted to the synapse connections. If the weights of the synapse connections are varied, configuration of the network is changed, and thus, the network yields different output values in response to identical input data.
Meanwhile, in this neural network, when data belonging to one of two events, having a predetermined interrelation, are applied to the units 5 of the input layer 1 and data (teacher's data) corresponding to the input data and belonging to the other event are applied to the units 7 of the output layer 3, the neural network performs a learning process based on error back propagation. Thus, the neural network resets the values of the weights of the synapse connections so as to reconfigure the network such that data identical with the above mentioned teacher's data are outputted from the units 7 of the output layer 3. In this neural network upon learning, when arbitrary data belonging to the above described one event are inputted to the units 5 of the input layer 1, data corresponding to the input data and belonging to the other event are outputted from the units 7 of the output layer 3.
In the three-layer perceptron type neural network referred to above, the number of the units 5 of the input layer 1 is determined by the degree of values of the input data, while the number of the units 7 of the output layer 3 is determined by the number of categories to be recognized. However, the number of the units 6 of the intermediate layer 2 varies according to the number of the units 7 of the output layer 3, accuracy of recognition, processing time, etc., and therefore, is determined by trial and error.
In the known three-layer perceptron type neural network, the number of the units 6 of the intermediate layer 2 is determined bY trial and error as described above, and therefore, is usually set at a value larger than an expected necessary number in view of accuracy of recognition, etc. However, if the number of the units of the layers is increased, the number of connections between the units is also increased, thereby resulting in an undesirable increase in computing time for the neural network. Furthermore, if the number of the units is increased, units which are not associated with learning, in other words, units which are not associated with recognition are produced, thus resulting in deterioration of efficiency of learning and recognition.